1. Field of the Invention
The invention relates to the field of transferring data between computers using network resources and more particularly to controlling access to the network resources.
2. Relevant Background
Many internet service providers (isps) use blacklists to control access to their servers and services. Rbl (realtime blackhole list) is a list of ip adresses whose owners refuse to stop the proliferation of http://www.webopedia.com/term/r/spam.htm spam. The rbl usually lists server ip addresses from isps whose customers are responsible for the spam and from isps whose servers are hijacked for spam relay.
As subscribers to the rbl, isps and companies will know from which ip addresses to block traffic. This type of traffic blocking occurs prior to the smtp (simple mail transfer protocol,) connection phase. The receiving end will check the rbl for the connecting ip address. If the ip address matches one on the list, then the connection gets dropped before accepting any traffic from the connecting client. Some isps though will choose to blackhole (or ignore) ip packets at their routers. The goal here is to block all ip traffic from clients on the rbl blacklists.
It is important to note that the recipient end, not the rbl administrator, does all e-mail and packet blocking which may choose to honour lists that indicate spam may be directed at its servers.
Mail abuse prevention system (maps) 11c created the rbl, but there are other entities that keep rbls aside from maps.
Blacklists are provided by a variety of sources, and may be used to control many different types of services, including but not limited to websites, data servers, games servers, instant messaging systems, voice over internet protocol (voip) phones and electronic mail services. The reasons for a particular ip address being on a blacklist are arbitrarily set by whoever controls the rbl service. A typical reason may be that the address has previously been reported for an action that has been deemed offensive, i.e. An attempt to illegitimately access a service (hacking) or the address being identified as a source of spam email. Many service providers regard the ability to block known offenders as crucial for effective operation of their services.
The ability to determine the ip address that the reported offender is coming from can occur at several levels. It can happen at the operating system or kernel level, for example in linux this can be controlled at the ‘iptables’ level, or it can happen via a tcp connection server wrapper, ie via ‘tcpserver’ or ‘xinetd’ before it is passed on to the application service, or it can happen at the service level, i.e. The webserver, dns or smtpd process.
There are several limitations related to the current usage of rbl's. Typically, rbl lists are polled by a lookup program run by the isp to determine whether a specific ip address is listed. The rbl may be maintained at remote offsite locations and may be queried via a udp/ip or tcp/ip type connection to the rbl.
One limitation id due to the format of an ip address, which is a 32-bit numeric address written as four numbers separated by periods. Each number can be zero to 255. For example, 1.163.15.240. Due to the size of the ip address space blacklists are often too big for efficient local storage (256×256×256×256 addresses for ipv4 (ip version 4) and even more so for ipv6 (ip version 6) adresses. Furthermore, disseminating these lists from their collection points across networks or internets becomes onerous and bandwidth intensive and many services that need access control the most, don't have the ability to collect and/or store these lists.
Compilation services offer a central database of address. most blacklists are typically accessed via a live lookup against these central databases of addresses. this means that the rbl dns service is accessed for every connection initiation attempt by a user. There are numerous rbl services, some that may be accessed free and some that charge for access. examples of well known compilation services may be accessed at the following url's.                http://www.spews.org        http://www.spamcop.net        http://www.spambag.org        http://www.ordb.org        
The use of these rbl database services has not been without problems. Because the dns/rbl database is queried for each user connection connectivity problems or network slowdowns can cause lookups to timeout and/or fail. Some rbl services have also incurred malicious denial of service (dos) attacks from hackers, preventing legitimate lookups for a period of time. Additionally when an rbl service goes offline, it may be that all incoming addresses could be rejected, causing an embarrassing or costly interruption of service. Legal challenges to rbl services have also been launched and in some cases it has been mandated that there must be an ability to override a blacklist entry. Most rbl systems do not have a contingency for overriding entries.
Another problem is that due to the lookup process it is not practical to access several rbl lists for each user connection. It would be highly desirable to be able to query multiple lists for each connection to ensure the broadest possible protection, since rbl providers may use very different inclusion criteria.
Additionally since blacklists from different providers are formatted differently, the lookup process may have to be individually customized for each provider further complicating the lookup process.
On average, a lookup using a commercial rbl system can take up to approximately 1-3 seconds or even longer, depending on network latency, which in the example of a commercial mail server processing over 1 million connections per day, is very inefficient. A common mail server has in the range of 200-400 simultaneous connections making it imperative that each connection occupies the minimum amount of time. Any type of network request for a lookup can result in serious delays. A remote rbl lookup to a database may encounter rbl service delays due to loading that could prevent the lookup from completing.
There remains a need for an rbl lookup process that mitigates at least some of the aforementioned problems.